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Background / Context: 

Description of prior research and its intellectual context. 

Teachers are generally recognized as the schooling factor accounting for the highest 
proportion of student learning outcomes (Aaronson et al., 2007; Hanushek, 1986). This implies 
the quick and dramatic improvement in school perfonnance observed in turnaround (TA) 
schools was associated with a major change in the perfonnance of its teachers. This change could 
be manifest in one of two ways — either the teachers in the school dramatically improve or 
previously ineffective teachers are replaced with effective teachers. This paper seeks to 
understand how each of these potential changes in the school’s human capital stock contributed 
to turnaround in Florida and North Carolina. 

Replacing teachers in a low-performing school is a key element of several of the ARRA 
school improvement models; yet, the efficacy of wholesale staff replacement is not well 
established (Herman et al., 2008). This model is at odds with naturally occurring patterns in the 
labor market in which qualified or effective teachers move away from low-performing schools 
(Boyd et al., 2009; Hanushek et al., 2004), implying that such a change in the pool of human 
capital is unlikely without some extraordinary intervention efforts. 

Likewise, improving the stock of current human capital appears to counter documented 
evidence on teacher perfonnance. Teacher performance appears to be generally (though not 
perfectly) stable over time, and changes in performance (for better or worse) appear to be 
transitory (Goldhaber and Hansen, 2010). Prior studies generally find only small improvements 
in a teacher’s perfonnance beyond the first few years of teaching (e.g., Rockoff, 2004), and no 
gains in human capital associated with the attainment of additional credentials such as an 
advanced teaching degree or national board certification (Goldhaber and Anthony, 2007). 
Moreover, prior empirical studies linking professional development to teacher productivity on 
student test scores are generally mixed and not rigorous enough to detennine whether 
professional development actually has a net positive impact on students (Wayne et al., 2008) and 
a recent evaluation of a professional development program using a randomized control design 
found no detectable effects associated with the program (Garet et al., 2011). 

In summary, both methods of affecting the pool of human capital in a school (either 
replacing it or improving upon it) appear similarly unlikely to occur without targeted 
intervention in a school. In evaluating how these two sources of improvement worked in TA 
schools in these states, this study cannot hypothesize a priori which of the two will dominate. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

This paper seeks to understand how two potential changes in each school’s teacher 
workforce contributed to turnaround in Florida and North Carolina. Specifically, the study 
investigates the following research question: is the improved achievement observed in successful 
turnaround schools most strongly associated with workforce development or turnover (or, some 
combination of the two)? 
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We utilize the Study I designations of chronically low-perfonning (CLP) and turnaround 
(TA) schools to investigate whether there are distinct differences that arise between TA and non- 
TA CLP schools in the teacher workforce. We relied on administrative data drawn from state 
longitudinal data bases, which include infonnation on teachers’ experience, credentials, and 
mobility. In addition, student-teacher linked test score data were used to calculate teachers’ 
value-added estimates of effectiveness (i.e., the mean improvement in student test scores 
controlling for other covariates). 

Setting: 

Description of the research location. 

(May not be applicable for research methods or panel submissions) 

Chronically low-performing elementary and middle schools identified in Florida and 
North Carolina, using state administrative data on student perfonnance spanning six school years 
(2002-03 through 2007-08). 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 

(May not be applicable for research methods submissions) 

The population of interest is the teacher workforce in schools identified in the study as 
chronically low perfonning. Key descriptors and characteristics of each of the study’s data 
samples are presented in Table 1. 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

(May not be applicable for research methods submissions) 

This study does not investigate a particular intervention, program, or practice. Rather, this 
study seeks to investigate whether the observed improvement in perfonnance in TA schools 
appears to come from workforce turnover (i.e., poor teachers exiting the school and effective 
teachers entering the school) or whether improvements appear to come from human capital 
development generally (i.e., improved performance observed across workforce generally). 

Research Design: 

Description of the research design. 

A difference-in-difference-in-difference model is used to predict student-level test scores. 
The three levels of differences come from: 1) the study’s pre- / post-period outpoint; 2) a 
comparison of TA vs. non-TA CLP schools; and 3) a comparison between teacher groups in the 
school workforce, differentiating teachers as outgoing (those observed in the pre -period only), 
incoming (those observed in the post-period only), and stable (those observed in both pre- and 
post-periods). The econometric model estimated is as follows: 

A jsl = A,, -A + x uPi + tap* + post a 

+ OUTGOING .p 5 + TA s * POSfp, + OUTGOING , * TAJI 1 
+ INCOMING j * POST, pi + INCOMING j * TA S * POST,p q + s ijst 
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In this model, current achievement on test scores for student i, linked with classroom 
teacher j, in school 5, at time t ( A ijst ) is the dependent variable. Explanatory student-level 

variables in the model are the student’s prior test scores in both reading and math ( A j ), current 
student characteristics including gender, race, eligibility for free or reduced-price lunch, special 
education status, limited English proficiency status ( X it ). The indicator variables used for the 

difference-in-difference-in-difference estimation strategy represent different schools ( TA S , with 
non-TA schools as the omitted reference group), different time periods ( POST t , with the pre- 
period omitted), and different groups of teachers ( OUTGOING j & INCOMING ; , with stable 
teachers as the reference group). 

Statistical, Measurement, or Econometric Model: 

Description of the proposed new methods or novel applications of existing methods. 

(May not be applicable to submissions in sections other than research methods) 

Not applicable. 

Usefulness / Applicability of Method: 

Demonstration of the usefulness of the proposed methods using hypothetical or reed data. 

(May not be applicable submissions in sections other than research methods) 

Not applicable. 

Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

(May not be applicable for research methods submissions) 

Data analysis on student-teacher linked administrative education data in two states. No 
supplemental data were collected. 

Findings / Results: 

Description of the main findings with specific details. 

(May not be applicable for research methods submissions) 

Table 2 presents the estimated coefficients from the primary model. Results are reported 
from both Florida (in columns 1-4) and North Carolina (columns 5-8), with specifications both 
excluding school random effects (odd-numbered columns) and those including school random 
effects (even-numbered). 

Three noteworthy findings emerge from the estimated regression coefficients. First, a 
general negative selection is observed among all CLP schools in most data samples in which 
outgoing teachers are associated with lower student achievement (excepting North Carolina 
elementary schools). Second, a general improvement is observed among all teachers (both stable 
and incoming) in TA schools during the post-period, supporting the hypothesis of overall 
productivity improvement in teachers. Third, with the exception of Florida middle schools, there 
is no significant difference in the productivity of incoming teachers in TA schools during the 


SREE Fall 2012 Conference Abstract Template 


3 



post-period, which suggests the recruitment of particularly effective teachers to CLP schools did 
not play a systematic role in explaining these schools’ perfonnance improvements. 

Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

The results suggest TA schools improved through a combination of improving the stable 
teachers who had been in the school even when it was performing poorly and bringing in 
relatively effective talent into the school. Note, however, that incoming teachers in TA schools 
during the post-period were in general not more productive than stable teachers in TA schools 
during the post-period (though they were more productive than incoming teachers in non-TA 
schools during the same period), giving little evidence that “talent transfer” played a role in 
improving these schools. 

It is important to note that the evidence presented here is descriptive only, and does not 
necessarily refute relying on a strategy of replacing staff to support school turnaround; however, 
it simply shows that improvements from the pool of stable teachers appear to account for 
considerable improvements in performance within these historical turnaround schools. 

Finally, it is important to note that while the model employed here attributes the 
differences in school performance over time to changes in the effectiveness of teachers in the 
school, both new and existing, these differences may not be due to teachers alone. The 
administrative data employed here are not sufficiently detailed to separate the role of the teacher 
from the role of the principal or other school-wide interventions. Findings from the Study’s 
survey and case study data may cast more light on human resource factors in the context of other 
refonn initiatives. 
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Not included in page count. 
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Appendix B. Tables and Figures 
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Table 1. Descriptive Statistics of Teachers and Students in Four Data Samples, by TA status 


State 

Florida 

North Carolina 

School sample 

Elementary 

Middle 

Elementary 

Middle 

School type 

TA 

Non-TA 

TA 

Non-TA 

TA 

Non-TA 

TA 

Non-TA 

Student Body Characteristics 









Proportion of female students 

SO. 7% 

50.7% 

53.5% 

49.9% 

48.9% 

51.1% 

50.4% 

50.4% 

Proportion of African American students 

54.9% 

46.4% 

42.8% 

40.0% 

57.9% 

60.9% 

62.4% 

72.0% 

Proportion of Hispanic students 

16.9% 

25.4% 

30.8% 

14.8% 

8.7% 

8.2% 

6.7% 

5.1% 

Proportions of parents holding BA degree or 
higher 





22.6% 

17.8% 

25.5% 

24.4% 

Proportion of students with limited English 
proficiency 

6.1% 

7.1% 

4.3% 

0.8% 

4.4% 

3.0% 

2.3% 

1.7% 

Proportion of students ever eligible for free or 
reduced-price lunch program 

88.4% 

87.6% 

82.9% 

74.2% 

74.3% 

75.0% 

67.1% 

76.5% 

Mean student achievement in math 
(standardized) 

-0.412 

-0.385 

-0.209 

-0.160 

-0.436 

-0.517 

-0.306 

-0.390 

Mean re-enrollment rate (2003 and 2004 
only) 

62.9% 

65.6% 

27.6% 

31.8% 

48.6% 

43.2% 

26.0% 

34.2% 

Mean school enrollment 

101 

102 

419 

433 

89 

81 

188 

177 

Teacher Characteristics 









Proportion of females 

83.2% 

83.7% 

69.6% 

72.9% 

89.0% 

94.0% 

82.3% 

82.0% 

Proportion of minority teachers (Currently 
classified as Black or Hispanic) 

38.5% 

40.9% 

46.4% 

33.9% 

33.3% 

26.0% 

46.6% 

47.4% 

Proportion of teachers with 4 or more years 
of experience 

58.8% 

61.0% 

54.0% 

79.6% 

74.8% 

72.0% 

69.8% 

64.9% 

Proportions of teachers fully licensed 





86.4% 

84.8% 

77.4% 

76.0% 

Mean proportion of teachers returning in 
following year (2003 and 2004 only) 

52.1% 

57.6% 

40.7% 

35.6% 

62.0% 

56.0% 

45.2% 

46.6% 

Total Schools 

17 

94 

3 

24 

8 

66 

5 

37 


Table 2. Regression estimates of difference-in-difference-in-difference estimators 


State 

Florida 

North Carolina 

School sample 

Elementary 

Middle 

Elementary 

Middle 

School Fixed Effects 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

TA 

0.002 

0.008 

-0.053** 

-0.065* 

-0.048** 

-0.028 

-0.015 

0.011 

1 A 

(0.013) 

(0.027) 

(0.014) 

(0.032) 

(0.017) 

(0.026) 

(0.015) 

(0.039) 


0.116** 

0.113** 

0.030** 

0.034** 

0.005 

0.010 

-0.037** 

-0.032** 

Post 

(0.008) 

(0.008) 

(0.007) 

(0.007) 

(0.007) 

(0.007) 

(0.007) 

(0.007) 


0.125** 

0.126** 

0.141** 

0.155** 

0.153** 

0.146** 

0.073** 

0.067** 

TA*Post 

(0.018) 

(0.018) 

(0.025) 

(0.026) 

(0.023) 

(0.023) 

(0.022) 

(0.022) 

Outgoing 

-0.017 

-0.026** 

-0.034** 

-0.039** 

0.018* 

0.016 

-0.031** 

-0.018* 

(0.009) 

(0.009) 

(0.008) 

(0.008) 

(0.008) 

(0.009) 

(0.007) 

(0.008) 

Outgoing*TA 

0.009 

0.026 

-0.037 

-0.024 

0.027 

0.011 

0.014 

-0.000 

(0.023) 

(0.024) 

(0.021) 

(0.023) 

(0.027) 

(0.028) 

(0.022) 

(0.022) 

lncoming*Post 

-0.048** 

-0.059** 

-0.025** 

-0.035** 

-0.015 

-0.034** 

-0.045** 

-0.037** 

(0.007) 

(0.007) 

(0.008) 

(0.008) 

(0.008) 

(0.009) 

(0.009) 

(0.009) 

lncoming*TA*Post 

0.017 

0.032 

-0.063* 

-0.062* 

0.003 

0.009 

0.051 

0.042 

(0.017) 

(0.017) 

(0.028) 

(0.030) 

(0.025) 

(0.026) 

(0.027) 

(0.027) 










Observations 

55,548 

55,548 

34,582 

34,582 

39,516 

39,516 

37,353 

37,353 

R-squared 

0.577 

0.576 

0.628 

0.628 

0.641 

0.641 

0.682 

0.682 
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